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Abstract 

In  this  paper  we  develop  approximation  algorithms  for  two-stage  convex  chance  constrained 
problems.  Nemirovski  and  Shapiro  [18]  formulated  this  class  of  problems  and  proposed  an 
ellipsoid-like  iterative  algorithm  for  the  special  case  where  the  impact  function  f  (x,  h)  is  bi-affine. 
We  show  that  this  algorithm  extends  to  bi-convex  f(x,  h)  in  a  fairly  straightforward  fashion. 
The  complexity  of  the  solution  algorithm  as  well  as  the  quality  of  its  output  are  functions  of  the 
radius  r  of  the  largest  Euclidean  ball  that  can  be  inscribed  in  the  polytope  defined  by  a  random 
set  of  linear  inequalities  generated  by  the  algorithm  [18].  Since  the  polytope  determining  r 
is  random,  computing  r  is  difficult.  Yet,  the  solution  algorithm  requires  r  as  an  input.  In 
this  paper  we  provide  some  guidance  for  selecting  r.  We  show  that  the  largest  value  of  r  is 
determined  by  the  degree  of  robust  feasibility  of  the  two-stage  chance  constrained  problem  - 
the  more  robust  the  problem,  the  higher  one  can  set  the  parameter  r. 

Next,  we  formulate  ambiguous  two-stage  chance  constrained  problems.  In  this  formulation, 
the  random  variables  defining  the  chance  constraint  are  known  to  have  a  fixed  distribution; 
however,  the  decision  maker  is  only  able  to  estimate  this  distribution  to  within  some  error.  We 
construct  an  algorithm  that  solves  the  ambiguous  two-stage  chance  constrained  problem  when 
the  impact  function  f(x,  h)  is  bi- affine  and  the  extreme  points  of  a  certain  “dual”  polytope  are 
known  explicitly. 


1  Introduction 

The  simplest  model  for  a  convex  chance  constrained  problem  is  as  follows, 
min  crx 

s.t.  x  €  *e(Q)  =  {y  €  V  |  Q(H  :  /( y,  H)  >  0)  <  e},  '  1  ' 
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where  x  E  Rn  is  the  decision  vector,  the  uncertain  parameter  H  £  Rm  is  distributed  according  to 
the  known  distribution  Q,  X  C  Rn  is  a  convex  set,  and  /(•,  H)  :  Rn  i— >  R  is  convex  in  x  for  each 
fixed  H.  Since  a  collection  of  convex  constraints  /;(x,  H)  <  0,  i  =  1, . . .  ,p,  can  be  formulated  as 
a  single  convex  constraint  /(x,  H)  =  maxi<j<p{/j(x,  H)}  <  0,  restricting  the  range  of  constraint 
function  /(•,  H)  to  R  does  not  represent  any  loss  of  generality.  Also,  by  introducing  a  new  variable 
if  necessary,  one  can  assume  that  the  objective  function  is  linear  and  independent  of  the  uncertain 
parameter. 

Chance  constrained  problems  are  computationally  very  difficult  to  solve.  Except  for  a  very 
restricted  class  of  measures,  evaluating  Q(H  :  /(x,  H)  >  0)  involves  numerically  computing  a 
multidimensional  integral  that  becomes  difficult  as  the  number  of  parameters  grows.  Moreover,  even 
if  the  function  /(x,  H)  is  linear  in  x,  the  feasible  set  Ae(Q)  of  (1)  is,  in  general,  not  convex.  General 
chance  constrained  problems  (i.e. ,  /(x,  H)  is  possibly  non-convex)  have  a  very  rich  literature. 
See  [19,  22]  for  a  survey  of  solution  methods. 

Recently  there  has  been  a  renewed  interest  in  the  special  case  of  convex  chance  constrained  prob¬ 
lems.  One  approximates  the  convex  chance  constrained  problem  (1)  by  the  sampled  optimization 
problem 

min  cTx  .  . 

(2) 

s.t.  xG^[H1,JV]={yG^|/(y,Hi)<0,i  =  l,..,iV}( 

where  H,;,  i  =  1, . . . ,  N,  are  N  IID  samples  from  Q.  de  Farias  and  Van  Roy  [9]  use  results  from 
Learning  theory  [1,  15,  27]  to  show  that  for  the  special  case  of  linear  constraints  a  sample  size  of 


4n 


12 


N  >  —  log  —  +  -  log  - 


ensures  that  the  set  of  decision  vectors  feasible  for  the  sampled  problem  (2)  is  contained  in  Ag(Q) 
with  a  probability  at  least  1  —  5.  Erdogan  and  Iyengar  [11]  show  a  similar  bound  for  general  convex 
constraints  with  the  constant  n  replaced  by  the  Vapnik-Chervonenkis  (VC)  dimension  df  of  the 
constraint  function.  Calafiore  and  Campi  [7,  8]  show  that  when 


N  >  —  In 
e 


2 

H —  In 
€ 


+  2n, 


the  optimal  solution  of  the  sampled  problem  (2)  is  feasible  for  (1)  with  a  probability  at  least 
1  —  5.  This  bound  is  particularly  relevant  since  the  VC  dimension  df  of  a  constraint  can  be  orders 
of  magnitude  larger  than  the  problem  dimension  n.  Note  that  these  results  only  provide  upper 
bounds  for  the  number  of  samples,  i.e.  only  a  sufficient  condition.  Thus,  a  natural  question  of 
the  quality  or  “tightness”  of  the  approximation  arises.  Recently,  Nemirovski  and  Shapiro  [18,  17] 
established  logarithmically  separated  upper  and  lower  bounds  on  the  number  of  samples  required  to 
approximate  a  convex  chance  constrained  problem  when  the  measure  Q  has  a  certain  concentration- 
of-measure  property. 
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The  chance  constrained  problem  assumes  that  the  distribution  (Q)  of  the  random  parameter  H 
is  known  and  fixed.  In  practice,  however,  the  distribution  Q  is  only  specified  with  some  error. 
Erdogan  and  Iyengar  [11]  model  this  ambiguity  in  the  measure  by  assuming  that  the  measure  Q  is 
only  known  to  belong  to  the  set 


Q  = 


<P}, 


where  pp(-,-)  denotes  the  Prohorov  metric  (see  §  4).  Given  this  description  of  the  information 
available  to  the  decision-maker,  the  ambiguous  chance  constrained  problem  is  given  by 


nun 


T 

C  X 


s.t.  X  G  X£  =  y  G  *  I  Q(H  :  /( y,  H)  >  0)  <  e,  VQ  G 
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(3) 


Although  the  problem  (3)  was  explicitly  introduced  in  Erdogan  and  Iyengar  [11],  the  minimax 
formulation  has  a  long  history  in  stochastic  programming  [28,  10,  25,  23,  24],  Motivated  by  the 
fact  that  the  sampled  problem  (2)  is  a  good  approximation  for  the  chance  constrained  problem  (1), 
Erdogan  and  Iyengar  approximate  (3)  by  the  robust  sampled  problem 


mm 


T 

C  X 


(4) 


s.t.  X  G  y[N,  (3]  =  {y  <E  X  I  /(y,  z)  <  0,  V  z  s.t.  ||z  -  H?||  <  (3,  i  =  l,...,N}, 

where  Hjjy  are  N  IID  samples  drawn  from  the  central  measure  Qo  and  ||  •  |j  is  the  norm  used  to 
define  the  Prohorov  metric  pp(-,-).  For  appropriately  chosen  norms,  such  problems  can  be  solved 
efficiently  using  the  technique  detailed  in  [6] .  The  following  approximation  results  were  established 
in  [11]. 

(a)  N  >  yz^log(jr^)  +  log  ( where  df  is  the  VC-dimension  of  the  function  class 
{/(x,  ■)  :  x  €  X}  and  p  =  2(|  +  log(/3  +  2_e/2)),  suffices  to  ensure  P(3;[A1, 0\  C  Afe(Q))  >1  —  5, 
for  any  fixed  Q  €  Q. 


(b)  N  >  jz ‘g  In  (jzrg)  +  In  (|)  +  2n  ensures  that  the  optimal  solution  x  of  (4)  satisfies  P( 


1  e-0 

Xf)  >  1  —  5. 


x  G 


The  model  (1),  while  quite  general  in  its  ability  to  model  constraints,  is  limited  to  the  so- 
called  single  stage  problems  where  decisions  must  be  made  before  the  uncertain  parameter  H  is 
revealed.  A  natural  extension  is  to  consider  two-stage  problems  where  one  has  to  commit  to  the 
first  stage  decision  x  before  the  realization  of  the  uncertain  parameter  H,  and  the  second  stage 
decision  variable  v  can  be  chosen  after  observing  the  parameter  H.  A  prototypical  example  of  a 
two-stage  problem  is  the  network  design  where  the  first  stage  variables  are  the  capacities  on  the  arcs 
and  the  second  stage  variables  are  the  routing  decisions.  The  simplest  two-stage  chance  constrained 
problem  is  given  by 


mm 

s.t. 


T 

C  X 


X  G  Xe(Q)  =  {y  G  A  |  Q(H  :  3v  s.t.  Wv  >  f  (y,  H))  >  1  -  e}  , 


(5) 
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where  the  impact  function  f(x,  h)  :  Rn  X  Rm  i— >  R(  is  bi-affine,  v  £  Rfc,  and  W  £  R/xfc.  Since 
the  matrix  W  does  not  depend  on  the  realization  of  H,  problems  of  the  form  (5)  are  said  to  have 
a  fixed  recourse.  Thus,  (5)  is  a  two-stage  linear  chance  constrained  problem  with  a  fixed  recourse. 
This  model  was  introduced  by  Nemirovski  and  Shapiro  [18]. 

One  could  attempt  to  approximate  (5)  by  the  sampled  two-stage  LP 

min  c1 x 

(6) 

s.t.  X  £  T[Hijv]  =  {y  £  T  I  3vj  s.t.  Wv;  >  f(y,H,),z  =  1 . . .  ,N}. 

However,  note  that  since  each  scenario  H,  has  its  own  set  of  second-stage  variables  Vj,  i  =  1, . . . ,  N, 
the  problem  dimension  grows  with  N,  and  the  results  of  Calafiore  and  Campi  [7,  8]  no  longer  yield 
a  good  bound  on  the  number  of  samples  required  to  produce  a  good  approximation.  Also,  since 
it  is  difficult  to  quantify  the  VC-dimension  of  the  two-stage  constraint,  the  results  in  [11]  cannot 
be  applied.  Nemirovski  and  Shapiro  [18]  constructed  an  iterative  solution  algorithm  for  (5)  that 
closely  resembles  the  ellipsoid  algorithm  [5,  16,  26]. 

Our  contributions  in  this  paper  are  as  follows. 

(a)  We  extend  the  iterative  algorithm  proposed  in  [18]  to  solve  two-stage  chance  constrained  prob¬ 
lems  where  the  impact  function  f(x,  h)  (see  (5)  for  details)  is  bi-convex,  i.e.  convex  in  one 
variable  when  the  other  variable  is  held  constant.  We  still  assume  a  constant  recourse  ma¬ 
trix  W.  This  fairly  straightforward  extension  is  discussed  in  §  3. 

(b)  Since  the  iterative  algorithm  proposed  in  [18]  closely  resembles  the  ellipsoid  algorithm,  the 
number  of  iterations  required  to  compute  a  feasible  solution  as  well  as  the  quality  of  the 
solution  are  functions  of  the  radius  r  of  a  ball  with  the  largest  volume  contained  within  the 
feasible  set  of  the  problem.  We  show  that  the  value  of  the  parameter  r  is  determined  by  the 
degree  of  robust  feasibility  of  the  chance  constrained  problem.  This  is  similar  to  the  results 
relating  the  condition  number  of  optimization  problems  to  the  complexity  of  solving  them  to 
optimality  [20,  21,  13].  This  result  is  proved  in  §  3  and  discussed  in  §  3.1. 

(c)  We  formulate  the  ambiguous  two-stage  chance  constrained  problem  and  modify  the  algorithm 
proposed  in  [18]  to  compute  a  good  solution  for  this  problem.  Our  extension  is  limited  to  the 
special  case  where  the  function  f(x,  h)  is  bi-affine  (i.e.  the  case  considered  in  [18])  and  the 
extreme  points  of  a  certain  “dual”  polytope  are  known  explicitly.  We  extend  all  the  known 
results  for  chance  constrained  problems  to  the  ambiguous  setting. 

The  rest  of  the  paper  is  organized  as  follows.  In  §  2  we  introduce  the  notation  that  we  use  in 
the  rest  of  the  paper.  In  §  3  we  discuss  the  two-stage  chance  constrained  problems  with  bi-convex 
impact  functions.  In  §  4  we  present  the  extension  to  the  ambiguous  two-stage  chance  constrained 
problems.  In  §  5  we  present  the  results  of  our  numerical  experiments  with  a  two-stage  network 
design  problem  on  a  very  simple  network.  In  §  6  we  discuss  the  results  in  the  paper  and  include 
some  concluding  remarks. 
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2  Notation 


Sets  will  be  denoted  by  calligraphic  letters,  e.g.  A,  and  Ac  will  denote  the  complement  of  the  set 
A.  All  (deterministic)  vectors  will  be  denoted  by  the  boldface  lowercase  letters,  e.g.  x.  Random 
vectors  and  samples  of  random  vectors  will  be  denoted  by  the  boldface  uppercase  letters,  e.g.  H, 
and  measures  will  be  denoted  by  the  mathematical  boldface  letters,  e.g.  P.  We  will  denote  that  a 
random  vector  H  has  distribution  Q  by  H  ~  Q  and  a  cr-algebra  on  a  space  Ti  by  !F(TC). 

The  norm  ||  •  ||  will  denote  the  usual  Euclidean  norm  ||x||  =  V x1  x  =  xf.  Br( y)  will  denote 

a  Euclidean  ball  of  radius  r  centered  at  y,  i.e.  B,,(y)  =  {x  :  ||x  —  y||  <  r},  and  the  set  Br  will 
denote  a  Euclidean  ball  of  radius  r,  i.e.  Br  =  Br(y)  for  some  fixed  y.  Given  an  input  x  6  Rn,  a 
separation  oracle  S A  for  a  convex  set  A  C  Rn  returns  an  affine  function  L  :  Rn  i— >  R  satisfying 


-SU(x) 


L  s.t.  L( z)  <0,  Vz  E  A.  L(x)  >0  x  0  A, 

L  =  0  otherwise 


The  gradient  of  a  function  L  :  Rn  i— >  R  will  be  denoted  by  VT. 


3  Approximating  two-stage  convex  chance  constrained  problems 

In  this  section  we  develop  an  approximation  algorithm  for  the  two-stage  convex  chance  constrained 
problem 

min  cTx 

s.t.  X  €  £(Q)  =  {y  E  A  |  Q(H  :  f(y,  H)  0  C)  <  e}, 
where  C  denotes  the  polyhedron 

C  =  {z  E  R*  |  z  <  Wv,v  E  Rfc},  (8) 

x  E  Rn  is  the  decision  vector,  H  E  Ti  is  a  random  parameter  vector  distributed  according  to  Q, 
W  E  R;xfc  is  a  recourse  matrix,  and  f(x,  h)  :  X  x  R;  is  an  impact  function.  Nemirovski  and 
Shapiro  [18]  introduced  the  particular  form  for  the  set  C  and  the  associated  chance  constrained 
problem  (7).  To  reiterate,  the  variable  x  denotes  the  first  stage  decisions  and  the  variable  v  E  Rfc 
denotes  the  second  stage  decisions.  We  assume  that  the  impact  function  f(x,  h)  and  the  sets  X 
and  Ti  satisfy  the  following  conditions. 

Assumption  1 

(i)  The  sets  X  and  Tt  are  convex  compact  sets.  Let  ||x||  <  Rx  (resp.  ||h||  <  R^)  for  all  x  E  X 
(resp.  h  E  Ti). 

(ii)  The  impact  function  f(x,  h)  is  bi-convex,  i.e.  for  all  fixed  h  E  Tt  (resp.  x  E  X),  the  function 
f(*,  h)  :  X  i  ^  R;  (resp.  f(x,  •)  :TL  i— ►  R^j  is  a  convex  function. 
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(Hi)  f(x,  h)  =  f°(x)  +  f1(x,  h),  where  f1(x,  ah)  =  af1(x,  h),  for  all  a  >  0,  (x,  h)  G  X  x  TL. 

(iv)  There  exists  a  non- decreasing  (finite-valued)  function  K y  :  R+  i— >  R+  such  that 

|| f (xi ,  h)  —  f(xo,  h) ||  <  iv^(||h||)||xi— xo||,  for  all  h  €  TL  and*. o,  xi  e  X.  Let  =  R^(Rh). 

(v)  There  exists  a  non- decreasing  (finite-valued)  function  K*  :  R+  i— >  R+  such  that 

||f(x,  hi)— f(x,  ho)  ||  <  if*(||x||)||hi  — ho ||,  for  all  hi,  ho  G  TL  and  x  €  X.  Let  K*  =  Kjf(Rx). 

The  condition  (i)  is  not  essential  for  the  results  to  hold  and  is  almost  always  satisfied  in  practice. 
Assuming  f(x,  h)  is  convex  in  x  for  all  fixed  h  G  TL  is  necessary  to  ensure  that  the  sampled  version 
of  (7)  is  tractable.  The  assumption  that  f(x,  h)  is  convex  in  h  for  all  fixed  x  G  X  and  has  a  com¬ 
ponent  that  is  homogeneous  allows  one  to  leverage  the  concentration-of-measure  property  defined 
below  in  Definition  1.  The  assumption  that  f(x,  h)  is  Lipschitz  continuous  individually  in  each 
variable  is  sufficient,  though  not  necessary,  to  establish  the  approximation  results.  Assumption  1 
may  appear  overly  restrictive;  however,  there  are  many  function  classes  that  satisfy  these.  Two 
canonical  examples  are  as  follows. 

(a)  Affine  constraints:  f(x,  h)  =  Aox  +  hiA.{x  with  X  C  Rn,  TL  C  Rm,  and  A*  G  Rixn 

for  %  =  0,  The  growth  functions  /\^(||h||)  =  0(l)(|||Ao|||  +  ||h||  |||Aj|||)  and 

K'J  (||x||)  =  0(1 ) ||x||  j  1 1 1 A* |||,  with  the  constants  depending  on  the  particular  choice  of  the 

vector  norm  ||  •  ||  and  the  matrix  norm  |||  •  |||,  satisfy  Assumption  1. 

(b)  Second-order  cone  constraints:  Each  component  of  f(x,  h)  is  a  conic  quadratic  representable 
function  [5],  e.g.  fj(x,  h)  =  sj (TAjX  +  p)T(TAjX  +  p)  —  tjx  —  Vj  with  X  C  Rn,  A j  G  Rfcxn, 
t j  G  Rn,  Vj  G  R  for  j  =  1, . . . ,  l,  and  TL  =  {h  =  (T,  p)  :  T  G  Rpxfc,  p  G  Rp,  }.  In  this  case, 
we  can  set  A'^(||h||)  =  0(1) (| | |T| 1 1  +  ||h|||||A|||)  and  AT^(||x||)  =  0(l)||x|||||A|||,  where  T  = 

A  =  [Ai;  A2; . . . ;  A,]. 

We  assume  that  Q  satishes  the  (9 ,  if)- concentration  of  measure  property  defined  as  follows  [18]. 

Definition  1  ([18])  Let  9  G  (|,  1]  and  if  {a,  9)  :  [1,  oo)  x  (9, 1]  >—>■  R+  be  a  convex,  non- decreasing 
and  non- constant  function  of  a.  A  distribution  Q  on  Rm  is  said  to  have  (9,  if) -concentration  of 
measure  property  if  for  all  a  >  1  and  closed  convex  sets  B  with  Q(£>)  >  9  >  6, 

Q({H  0  aB })  <  exp  {-if  (a,  9)}. 

This  assumption  essentially  states  that  a  small  “blow-up”  of  the  set  B  with  a  measure  of  at  least  9 
increases  its  measure  exponentially.  The  prototypical  example  of  a  measure  satisfying  such  a 
property  is  the  multivariate  Normal  distribution,  A7(0, 1)  it  satisfies  the  concentration  property 
with  if(a,9)  =  1(9)2,  where  $(•)  denotes  the  CDF  of  a  A7(0, 1)  random  variable.  The 

assumption  that  the  impact  function  has  a  homogeneous  component  (see  Assumption  1  part  (iii) ) 
is  made  to  leverage  the  concentration  property  of  the  measure  Q. 
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As  discussed  in  §  1 ,  a  convex  chance  constrained  problem  is  approximately  solved  by  computing 
a  solution  to  an  appropriately  defined  sampled  problem.  Erdogan  and  Iyengar [11]  (see  also  [9]) 
compute  bounds  for  the  number  of  samples  required  to  reliably  produce  a  solution  for  the  chance 
constrained  problem  when  the  VC-dimension  of  the  constraint  function  is  known.  Since  the  VC 
dimension  of  the  constraint  defining  (7)  is  difficult  to  quantify,  VC-dimension  based  bounds  are 
not  useful  in  solving  (7).  Calafiore  and  Carnpi  [7,  8]  bounded  the  number  of  samples  required 
in  terms  of  the  number  of  decision  variables  when  all  the  constraints  are  convex  in  x  for  any 
fixed  h.  Since  we  need  the  second  stage  variables  v  to  check  feasibility  for  each  sample,  the 
number  of  decision  variables  grows  linearly  with  the  number  of  samples;  this  renders  the  bounds 
in  [7,  8]  worthless.  We  propose  to  approximately  solve  (7)  using  Algorithm  SolveChance  shown 
in  Figure  1.  SolveChance  is  a  simple  extension  of  an  algorithm  proposed  by  Nemirovski  and 
Shapiro  [18]  to  solve  the  special  case  with  bi-afhne  impact  functions  f(x,  h).  The  extension  to 
the  bi-convex  case  is  fairly  straightforward;  our  main  contribution  is  to  show  that  feasibility  of 
an  appropriately  defined  conservative  version  of  (7)  implies  that  SolveChance  returns  a  “good” 
solution  with  a  high  probability.  Next,  we  carefully  describe  the  algorithm  and  then  prove  a  series 
of  intermediate  results  that  are  needed  to  establish  the  main  result  . 

SolveChance  uses  two  oracles,  Sx  and  Sfc.  The  oracle  Sx  is  the  separation  oracle  for  the 
convex  compact  set  X  and  the  oracle  ^(xjh),  for  a  fixed  h  £  Rm,  returns  a  linear  inequality 
L  :  Rn  i— >  R  that  separates  x  £  R"  from  the  convex  set 

77  =  {x  |  f(x,  h)  £  C}  =  {x  |  3v  £  RA  s.t.  Wv  >  f(x,  h)}.  (9) 

Thus,  x  £  77  if,  and  only  if,  the  value  of  the  optimization  problem 

min  ||  u  —  x||  (10) 

s.t.  Wv  —  f(u,  h)  >0 

is  equal  to  0.  When  the  optimal  value  of  (10)  is  strictly  positive,  any  sub-gradient,  d  at  the  optimal 
solution  u*  satisfies  dr(u  —  x)  >  0  for  all  u  £  77  and,  therefore,  serves  as  a  separating  hyperplane. 
Note  that  (10)  is  a  convex  optimization  problem  and  can  be  solved  very  efficiently  when  f(x,  h)  is 
an  affine  function  or  a  conic  quadratic  representable  function  [5].  The  above  discussion  establishes 
that  the  separation  oracle  Sti(x;  h)  is  well-defined. 

We  will  call  an  output  x  of  SolveChance  well-defined  if  x  0.  Nemirovski  and  Shapiro  [18] 
established  the  following  result  (see  also  Theorem  4). 

Theorem  1  Suppose  the  output  x  of  SolveChance  is  well-defined,  (i.e.  x  ^  0).  Then  x  is 
infeasible  for  (7),  i.e.  x  fL  Ae(Q),  with  a  probability  at  most  M6. 

Note  that  Theorem  1  does  not  consider  the  case  x  =  0.  Next,  we  compute  an  upper  bound  on  the 
probability  that  the  output  x  =  0.  Let  1  =  {L\, . . . ,  Lmn}  denote  an  ordered  list  of  the  MN  linear 
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Figure  1:  Algorithm  SolveChance 


inequalities  generated  by  the  calls  to  the  oracle  S-r.  over  the  course  of  one  run  of  SolveChance. 
Let  Xj  denote  the  convex  compact  set 

Xx  =  {x  G  A  :  Lj-(x)  <  0 ,j  =  1, . . . ,  MN}.  (11) 

Note  that  the  set  1  and,  therefore,  Xj  depend  on  the  IID  samples  where  each  Hj  ~  Q. 

The  following  lemma  was  stated  in  [18]  without  a  proof. 

Lemma  1  Suppose  the  set  Xx  contains  a  Euclidean  ball  Br  of  radius  r.  Then  the  output  x  of 
SolveChance  is  well  defined,  i.e.  x  /  0. 

Proof:  We  will  prove  the  result  by  contradiction.  Suppose  SolveChance  returns  x  =  0.  Let 
{(xt_i,dt)  :  t  =  1  ,...,M}  denote  the  iterates  and  the  separating  hyperplanes  generated  during 
one  run  of  SolveChance.  Since  x  =  0,  for  each  Xf,  t  =  0, . . . ,  M  —  1,  either  xj  0  X  or  there  exists 
some  j  G  {1, . . . ,  N(t  +  1)}  such  that  Ljfxt)  >  0.  Thus,  xt  0  Xx  for  all  t  =  0, . . . ,  M  —  1. 

By  the  definition  of  Xx,  it  follows  that  {(xt_i,dt)  :  t  =  1  is  a  set  of  iterates  and 

separating  hyperplanes  that  could  have  been  generated  while  using  the  Ellipsoid  algorithm  to  solve 
the  convex  optimization  problem  min{cTx  :  x  G  Xx}.  Since  Xf  0  Xx  for  all  t  =  0, . . . ,  M  —  1,  it 
follows  that  the  Ellipsoid  algorithm  returns  an  empty  solution.  This  is  a  contradiction  because  of 
the  choice  of  the  iteration  count  M  [5].  ■ 

For  a  >  1  and  r  >  0,  define 

Xe(a,r )  =  {x  G  |q(h  :  f  (x,  aH)  G  >  1  -  e}  , 

8 


(12) 


where 


A  7  =  {y  £  A  |  y  +  u  £  .4,  for  all  ||u||  <  7},  (13) 

denote  the  interior  7-ball  of  the  set  A.  Recall  that  Ry  =  Ky(RR)  is  the  maximum  value  of  the 
growth  function  Ky  ( • ) .  ^Froiri  (13),  it  follows  that 

f(x,  h)  £  77  f(x,  h)  +  u  €  C,  Vu  :  ||u||  <  fi.  (14) 

Since  f(x,  h)  £  C  if,  and  only  if,  there  exists  a  v  £  Rfc  such  that  Wv  >  f(x,h);  we  have  that 
f(x,  h)  £  C~ ^  if,  and  only  if,  for  all  u  with  ||u||  <  /x,  there  exists  a  v  £  Rfc  (possibly  a  function  of 
u)  such  that  Wv  >  f(x,  h)  4-  u.  The  set  Xe(a,  r)  can  be  interpreted  as  the  set  of  decision  vectors 
that  are  robustly  feasible  for  the  chance  constrained  set  Afe(Q)  [4], 

Theorem  2  Fix  y  £  Xe(a,r).  Then  the  Euclidean  ball  Br( y)  C  Xj  with  a  probability  at  least 
1  -  MNe~^l~e\ 

Proof:  Let  fir  =  rRy.  Then  we  have  that 

1-e  <  Q(H:f(y,aH)6C-^), 

=  Q(a-1H:f°(y)+f1(y,H)£C-^).  (15) 

Let  Tty  =  {h  :  f°(y)  +f1(y,  h)  £  Then  (15)  and  the  concentration  property  of  Q  imply  that 

Q(7fy)  >  1  —  e_^(a,1_e)  provided  1  —  e  >  9. 

Since  y  £  X~r ,  it  follows  that  Br( y)  C  X,  and  Lipschitz  continuity  of  f  implies  that  for  all 
x  £  Br(y)  and  h  £  Tty, 

||f(x,h)  -  f (y,  h) ||  <  Ky(\\h\\)\\x  -  y ||  <  rKy(Rn)  =  /xr, 

i.e.  f(x,  h)  =  f  (y,  h)  +  u  for  some  u  with  ||u|j  <  /ir .  Since  y  £  Xe(a,  r),  it  follows  that 

f(x,  h)  £  C,  Vx  £  Br( y),  h  £  TLy.  (16) 

Suppose  H*  £  TLy ,  for  all  i  =  1, . . . ,  MN.  Then  (16)  implies  that  Br( y)  C  Xj.  Thus, 

{hg mat  :  ^  £  Tty,i  =  1, . . . ,  MN}  C  {hi, mat  :  Br( y)  C  Xj}.  (17) 

Consequently, 

QMN(UhMN  :  Br{ y)  %  XR  <  QMN((HhMN  :  H?:  £  Tty,  i  =  1, . . . ,  MN)C), 

<  MNQ(Tty)  <  MJVe^(“’1_£). 

■ 

Theorem  2  implies  the  following  result. 
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Corollary  1  Suppose  X£(a,r)  fi  0.  Then  the  output  x  of  SolveChance  is  well-defined  (i.e. 
x  0J  with  a  probability  at  least  1  —  M Ne~^a,1~e\ 

Proof:  Lemma  1  implies  that  the  solution  x  is  well  defined  if  the  set  Xx  contains  a  Euclidean  ball 
Br  of  radius  r;  and,  by  Theorem  2,  the  probability  of  this  event  is  at  least  1  —  M  Ne^^a,1~e\  M 
Corollary  1  establishes  that  the  output  x  of  SolveChance  is  well  defined  with  a  high  proba¬ 
bility  provided  the  chance  constrained  problem  (7)  is  robustly  feasible  and  the  measure  Q  has  a 
concentration  of  measure  property.  Next,  we  establish  a  bound  on  the  value  cTx.  We  will  call  a 
well-defined  output  x  (i.e.  x  0)  of  SolveChance  an  (a,  r,  ^-approximation  of  (7)  if 

cTx  <  inf  c rx  +  w.  (18) 

xGVe(a,r) 

This  definition  was  introduced  in  [18]. 

Theorem  3  Suppose  X£(a,r)  0.  Then  SolveChance  returns  an  (a,  r,w)- approximate  solution 

of  (7)  with  a  probability  at  least  (1  —  M Ne~^^a,1~e^)(l  —  MS). 

Proof:  Fix  k  >  0  and  choose  yK  G  X£(a,r)  such  that 

cTyK  <  inf  ctx  +  k.  (19) 

xeAe(a,r) 

By  Lemma  1,  x  is  well  defined  on  the  event  {Br( yK)  C  Xj}  and  by  Theorem  2  the  probability 
of  this  event  is  at  least  1  —  Combining  this  with  Theorem  1,  we  can  show  that 

SolveChance  produces  a  well-defined  output  x  that  is  feasible  for  (7)  with  a  probability  at  least 
(1  -  MJVe-^“-1-£))(l  -  MS). 

Next,  we  show  that  Br(yK)  C  Xx  implies 

cTx  <  min  cTx  +  u>.  (20) 

xG  Xx 

This  result  is  established  by  exploiting  the  close  resemblance  of  SolveChance  to  the  Ellipsoid 
algorithm.  We  closely  follow  the  analysis  of  the  Ellipsoid  algorithm  detailed  in  the  proof  of  Theo¬ 
rem  5.2.1  in  [5]. 

Let  xj  =  argminxg<Y;r{cTx}.  Let  (x*,  E*),  t  =  0, 1, . . . ,  M  —  1,  denote  the  iterates  generated  by 
SolveChance  .  Let  £t  denote  the  ellipsoid  St  =  { z  |  (z  —  x^)7  E^1(z  —  xj)  <  1}.  The  choice  of  the 
iteration  count  M  ensures  that  there  exists  v  <  1  and  z  G  Xx  such  that 

(i)  vol(£t)/rn  <  v  <  1, 

(ii)  y  =  x J  +  v{z  —  xj)  0  SM- 

Since  y  G  Xx  C  X,  it  follows  that  y  G  So-  Therefore,  there  exists  r  <  M  such  that  y  G  ST  but 
y  0  ST+ 1-  Then  it  follows  that  d]Ts  >  d!^xr  for  all  s  G  ££+\  H  ST,  and,  in  particular, 

dr  y  >  drxr-  (21) 


10 


We  claim  that  the  iterate  xr  G  V,  i.e.  it  is  one  of  the  candidate  points  for  computing  the  output  x. 
Suppose  this  is  not  the  case.  Then  we  must  have  that  xr  0  XT,  where 

XT  =  {x  G  X  |  Lj(x)  <  0,  j  =  1, . . . ,  N(t  +  1)}  D  Xj, 

and  the  separating  hyperplane  dT  must  satisfy  d^s  <  d^xT  for  all  s  G  XT.  Thus,  the  bound  (21) 
together  with  the  fact  that  y  G  Xx  C  XT  leads  to  a  contradiction.  Now,  the  analysis  in  the  proof 
of  Theorem  5.2.1  in  [5]  implies  that  cTxr  <  rniiixg^  c1  x  +  u>  and  the  bound  (20)  follows. 

Finally,  Br{ yK)  C  Xj  implies  minxeiyz:  cTx  <  cTyK,  which  together  with  (19)  imply  that 

c 1  x  <  min  cTx  +  u  <  c1  yK  +  ui  <  inf  cTx  +  k  +  u 
xGA'x  xGA”€(a,r) 

Since  n  >  0  was  arbitrary,  the  result  follows.  ■ 

3.1  Discussion  of  the  approximation  result 

Algorithm  SolveChance  has  three  tunable  parameters,  namely  <5,  uj  and  r.  Nemirovski  and 
Shapiro  [18]  study  the  effects  of  these  parameters  on  the  running  time  and  approximation  quality 
of  SolveChance.  While  the  parameters  5  and  u  have  a  well-defined  meaning,  the  parameter  r  is 
rather  ad-hoc  and  it  is  not  clear  how  to  set  its  value.  The  parameter  r  is  clearly  very  important 
for  the  performance  of  SolveChance:  the  iteration  count  M,  the  probability  that  the  output  x 
of  SolveChance  is  well-defined  and  feasible  for  (7)  (see  Theorem  1  and  Corollary  1),  and  the 
approximation  guarantee  on  the  output  x  (see  Theorem  3)  are  all  inversely  proportional  to  the 
parameter  r.  Since  the  set  Xj  is  random,  selecting  r  is  difficult.  Yet,  SolveChance  requires  r  as 
an  input. 

The  main  contribution  of  this  paper  is  to  provide  guidance  in  selecting  r.  The  iteration  count  M 
as  well  as  the  probability  that  a  well-defined  x  is  infeasible  are  inversely  proportional  to  r.  This 
suggests  that  r  should  be  set  as  high  as  possible.  For  any  fixed  a  >  0,  the  maximum  allowed  value 
r  of  r  is  limited  by  the  requirement  that  the  set  Xe(a,r )  (see  (12))  is  non-empty,  i.e.  the  set  of 
decisions  that  are  robustly  feasible  for  (7)  is  non-empty  [4].  Thus,  the  computational  complexity  of 
SolveChance  is  intimately  related  to  the  robust  feasibility  of  (7)  -  the  more  robust  the  chance 
constrained  problem,  the  easier  it  is  to  compute  a  feasible  solution.  This  is  similar  to  the  relationship 
between  the  complexity  of  computing  an  optimal  solution  of  a  conic  linear  program  and  its  condition 
number  [20,  21,  13].  Although  setting  a  high  value  for  the  parameter  r  induces  SolveChance  to 
efficiently  produce  a  feasible  solution,  it  results  in  a  weak  approximation  guarantee  (see  (18)). 

Let  p  =  (1  —  MS)(  1  —  M N denote  the  probability  that  the  output  x  is  well-defined 
and  feasible.  If  p  >  0,  then  T  =  — independent  replications  of  SolveChance  ensure  that 
with  a  probability  1  —  7  at  least  one  of  the  outputs  is  feasible.  The  requirement  p  >  0  implies  an 
upper  bound  on  M,  and  consequently,  a  lower  bound  r  on  r.  Thus,  it  follows  that  there  is  a  lower 
bound  on  the  achievable  approximation  guarantee.  Within  the  range  [r,  r\  one  can  trade-off  the 
optimality  with  efficiency. 
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4  Ambiguous  two-stage  chance  constrained  problems 

In  this  section,  we  extend  the  approximation  results  to  ambiguous  two-stage  chance  constrained 
problems  where  the  distribution  of  the  uncertain  parameter  H  is  not  completely  known;  instead, 
the  limited  knowledge  about  the  distribution  is  characterized  by  the  uncertainty  set 

Q  =  {Q-PP(Q,Qo)<fi}-  (22) 

The  metric  pp(-,  •)  denotes  the  Prohorov  metric  defined  as 

/5P(Qi,Q2)  =  mf{7:Q1(6)  <Q2(^)  +  7,V6g^(H)},  (23) 

where 

£>7  =  (x  £  X  :  inf  1 1  x  —  z  1 1  <  7I. 

I  Z  es  J 

Although  the  definition  appears  asymmetric,  pp  is  a  metric.  It  plays  an  important  role  in  prob¬ 
ability  because  it  metrizes  weak  convergence.  Moreover,  pp(Q i,Q2)  is  the  minimum  distance  “in 
probability”  between  random  variables  distributed  according  to  Q,,  i  =  1,2. 

The  assumption  here  is  that  the  uncertain  parameter  H  is  distributed  according  to  some  fixed 
distribution  Q  6  Q;  however,  the  decision  maker  can  only  estimate  the  distribution  to  within  the 
error  /3.  The  goal  is  to  compute  a  solution  x  that  performs  “well”  for  all  distributions  in  the  set  Q. 
We  will  characterize  the  details  of  the  approximation  later  in  this  section.  We  make  the  following 
additional  assumptions  on  the  function  f(x,  h). 

Assumption  2 

(a)  The  function  f(x,h)  is  bi-affine,  i.e.  f(x,  h)  =  ao(x)  +  Ai(x)h,  where  the  vector  ao(x)  and  the 
matrix  Ai(x)  are  affine  functions  ofx. 

(b)  The  extreme  points  {A1^  :  i  =  1, . . .  ,p}  of  the  polytope  {A  :  WTA  =  0,  l1  A  =  1,  A  >  0}  are 
explicitly  known. 

These  constraints  are  quite  restrictive  and  we  will  comment  on  them  in  §  5. 

The  aim  of  this  section  is  to  establish  that  Algorithm  SolveAmbChance  displayed  in  Figure  2 
produces  a  solution  x  that  performs  “well”  for  all  Q  £  Q.  The  structure  of  SolveAmbChance  is 
very  similar  to  SolveChance,  with  the  following  two  distinctions.  First,  the  number  of  samples  N 
per  iteration  is  given  by  N  =  hi  Q)]  (contrast  with  N  =  [^lnQ)]);  thus  ambiguity  requires 
us  to  generate  more  samples  per  iteration.  And  second,  instead  of  S-ji,  Algorithm  SolveAm¬ 
bChance  employs  the  oracle  Stzp  that  is  a  separation  oracle  for  the  set  (for  a  fixed  h) 

n,3  =  jx  |  f(x,z)  G  C,  V  z  s.t.  || z  —  h||  <  /3 j, 

=  |x  |  Vz  s.t.  ||z  —  h||  <  /?,  3v  s.t.  Wv  >  f(x,  z)|.  (24) 
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Input:  e  >  0,  6  €  (0, 1),  separation  oracles  Sx  and  Stzb 

Output:  x 

set  V  +-  0,  N  =  In  Q)j ,  M  =  \2 n2  ln(^»  +  2)] 
set  xo  <—  0,  Eo  <—  Rxl 
for  t  =  1,  . . . ,  M  do 

Construct  a  direction  vector  dt 
st  Sx(*-t- 1) 
if  xt_i  0  X,  set  dj  <—  Vst 
else 

generate  Hi  jv  HD  Qo 

L N{t— l)+i  *  ip^t—  1 5  ,  Z  1,  .  .  .  ,  iV . 

if  3  j  E  {1, . . . ,  AT}  such  that  Lj(xt-i)  >  0,  set  dt  <—  VLj 
else  set  dt  =  c  and  V  *—  V  U  xt_i 

Given  (xf_i,Et_i)  and  dt,  set  (xf,Et)  by  the  Ellipsoid  method  update 
if  V  =  0  return  x  <—  0;  else  return  x  <—  arginin^-p-fc^  x} 

Figure  2:  Algorithm  SolveAmbChance 

The  set  7 Zp  has  the  same  structure  as  the  feasible  set  of  an  adjustably  robust  linear  program  [3]. 
It  is  well-known  that  checking  membership  in  such  a  set  is  NP-Complete  when  the  underlying 
polytope  is  described  by  a  set  of  inequalities  [3].  Since  we  assume  (see  Assumption  2 -(b))  that 
the  extreme  points  of  the  underlying  polytope  are  explicitly  available,  membership  in  TZg  can  be 
checked  efficiently.  From  (24),  it  follows  x  E  TZg  if,  and  only  if,  for  all  z  satisfying  |jz  —  h  II  <  P, 

0  <  Pxz  =  max  6  (25) 

s.t.  Wv  —  f (x,  z)  >  16. 

It  is  easy  to  check  that  Pxz  is  always  feasible.  In  order  to  construct  the  separating  hyperplane,  we 
consider  the  following  two  cases. 

(i)  There  exists  y  such  that  Wy  >  0.  In  this  case,  x  E  TZg  for  all  h.  Thus,  Stz0  (x)  =  0. 

(ii)  There  does  not  exist  y  such  that  Wy  >  0.  Then  Pxz  is  bounded;  thus,  by  strong  duality, 
there  is  no  duality  gap,  i.e. 

Pxz  =  min{— (f(x,z))TA  :  WTA  =  0, 1TA  =  1,  A  >  0}, 

=  min{—  (f(x,  z))TA(,')  :  i  =  1, . . .  ,p}, 

where  {A^  :  i  =  1, . . .  ,p}  are  the  extreme  points  of  the  dual  polytope.  Recall  that  we  have 
assumed  that  {A^}  are  explicitly  known. 
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Thus,  x  G  IZ/3  if,  and  only  if, 

0  <  min {Pxz  :  ||z  —  h||  <  /?}, 

=  min  |-(a0(x)  +  Ai(x)h)2A(,) -/9||Ai(x)tA(') ||)  . 
l<i<p  l  J 

Suppose  the  minimum  above  is  strictly  negative.  Let  k  =  argmin1<i<p  {  — (ao(x)+Ai(x)h)TA^  — 
/?||Ai(x  )tAW||}.  For  a  fixed  h  consider  the  convex  function  g( u;  h)  :  R"  i— >  R  defined  as 
follows 

5(u;h)  =  (a0(u)  +  Ai(u)h)TA(/,)  +  (3\\ Ai(u)t  A(A  1 1|. 

Then  any  sub-gradient  of  the  function  g(-;  h)  at  u  =  x  serves  as  the  separating  hyperplane. 

The  following  result  extends  Theorem  1  to  the  ambiguous  setting.  Note  that  for  all  the  results  in 
this  section  the  relevant  probability  measure  is  the  product  measure  ,  since  all  the  samples 
are  drawn  independently  from  the  central  measure  Qo- 

Theorem  4  Suppose  the  output  x  of  SolveAmbChance  is  well-defined,  i.e.  x  0.  Then,  for 
every  fixed  Q  G  Q,  we  have  that  Q(H  :  f(x,  H)  0  C)  >  e  with  a  probability  at  most  MS. 

Proof:  Fix  a  measure  Q  G  Q  and  let  Ae(Q)  =  {x  G  X  \  Q(H  :  f(x,  H)  0  C)  <  e}.  By  construction, 
the  events  {x  /  0}  =  U^Q1{xt  G  V}.  Consequently, 

{X  +  0}  n  {X  0  Ae(Q)}  C  U^oHixt  £P}n  {xt  0  Ae(Q)}).  (26) 

Fix  t.  Let  Bt  denote  the  event  that  xj_i  satisfies  all  the  N  inequalities  generated  by  the  oracle  Sji(j 
at  iteration  t.  Define 

yt[N,p]  =  {x  G  A  I  f (x,  z)  G  C,  V  z  s.t.  ||z  -  H°||  <  /?,  i  =  N(t  -  1)  +  1, . . . ,  Nt}  ,  (27) 

where  Nt  denote  N  IID  samples  drawn  according  to  the  central  probability  measure  Qo 

at  iteration  t.  Then  it  is  clear  that  the  event  Bt  =  {xj_i  G  yfiN,  /?]}. 

Let  At  denote  the  event  that  the  iterate  x*_i  satisfies  all  the  N(t  —  1)  inequalities  generated  by 
the  oracle  S-jz^  before  iteration  t.  Then  it  is  clear  that  {xt_i  G  V}  =  At  n  Bt.  Thus, 

Qf  (At  nBtn  {xf_!  0  Xe(Q)})  <  Q^(Bt  n  {xt__!  0  Ae(Q)}), 

<  Qof  {Bt  I  {xt_!  £Ae(Q)), 

<  S,  (28) 

where  the  bound  (28)  follows  from  Theorem  6  in  [11],  The  result  follows  applying  the  union  bound 
to  the  expression  in  (26).  ■ 

As  before,  define 

Xj  =  {x  G  X  :  Ljfx.)  <  0 ,j  =  1, . . .  ,MN}, 
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where  Lj,  j  =  1 , ,MN,  denote  the  set  of  linear  inequalities  generated  by  the  oracle  Sn0  over  the 
course  of  Algorithm  SolveAmbChance.  Then  a  simple  extension  of  the  technique  used  to  prove 
Lemma  1  establishes  the  following. 

Lemma  2  Suppose  the  set  Xj  contains  a  Euclidean  ball  Br  of  radius  r.  Then  the  solution  x 
returned  by  the  SolveAmbChance  is  well  defined,  i.e.  x/  0. 

For  a  >  1,  r  >  0,  and  a  fixed  measure  QgQ,  let 

Xe(Q,  a,  r)  =  {x  £  X~r  |Q(H  :  f (x,  aH)  G  C~^)  >  1  -  e  }  ,  (29) 

where 

Hr  =  («  +  1  )pKf  (. Rx )  +  rl<y  (Rh  +  p) .  (30) 

The  set  Xe(Q,a,r)  denotes  the  set  of  points  that  are  robustly  feasible  for  the  chance  constraint 
corresponding  to  the  measure  Q. 

Theorem  5  Fix  y  £  Xe(Q,a,r).  Then  the  Euclidean  ball  Br( y)  C  Xj  with  a  probability  at  least 
Proof:  Let  Hr  =  (a  +  1  )PKy  (Rx)  +  rKy^R^  +  /?).  Then,  we  have  that 


< 

Q(H  :  f(y,  aH)  £  C_/ir), 

(31) 

< 

P  +  Qo(H  +  u  :  ||u||  <  P,  f  (y,  aH)  £  C~^), 

(32) 

< 

P  +  Q0(H  :  f (y,  aH)  £  (R*}), 

(33) 

= 

P  +  Qo(a_1H  :  f°(y)  +  f1(y,H)  £  c~^+af3Kf{Rx)), 

(34) 

where  the  inequalities  (31)  and  (32),  respectively,  follow  from  the  definitions  of  y  and  the  Prohorov 
metric,  and  the  inequality  (33)  follows  from  the  Lipschitz  continuity  of  the  function  /. 

Let  Tty  =  {h  :  f°(y)  +  f1(y,  h)  £  ^~Ur+a/SKf  (Av)j.  qqien  (34)  ancj  ^he  concentration  property 
of  Qo  imply  that  Qo(^fy)  >  1  —  provided  1  —  e  —  P  >  9.  By  Assumption  1  we  have 

C  |h  :  f(y,z)  £C"^+(q+1)/3A'/’(^),Vz  s.t.  ||z-h||</?}.  (35) 

Thus,  for  all  h  G  T~ty  Sind  x  G  (y ) 5  Assumption  1  implies  th&t  for  nil  z  satisfying  ||z  h.||  ^  (3^  we 

have  that 

j|f(x,z)-f(y,z)||  <  A'^(||z||)||x-y||, 

<  ||h||+/3), 

<  rKy(Rn+fi).  (36) 

Since  h  £  (35)  and  (36)  imply  that  for  all  z  satisfying  ||z  h  ii  <p, 

f(x  z)  £  Gi-+(a+ (Rx)+rI<y(R-H+H)  =  q0  _  q 
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Consequently, 


Qon(*Ii,mn  ■  My)  2  x x) 

<  QoIN ((H-imn  ■  f(x,z)  G  C,Vx  G  Br(y),Vz  s.t.  ||z  -  Hj||  <  /3,i  =  1, . .  .,MN)C), 

<  MNQo(Hy)  <  MNe-^01'1-*-®. 


The  following  corollary  establishes  that  the  output  of  SolveAmbChance  is  well-defined  with  a 
high  probability. 

Corollary  2  Suppose  Xe(Q,a,r)  0.  Then  the  output  x  of  SolveAmbChance  is  well-defined 
(i.e.  x  0J  with  a  probability  at  least  1  —  MNe~^a,1~e~^ . 

Proof:  Lemma  2  implies  that  the  solution  x  is  well  defined  if  the  set  Xx  contains  a  Euclidean  ball 
Br  of  radius  r  and  the  probability  of  such  event  is,  by  Theorem  5,  at  least  1  —  M Ne~^^a,1~e~^ .  M 

Remark  1  By  setting  (i  =  0,  we  recover  the  corresponding  “unambiguous”  versions  of  Theorem  4 
and  Corollary  2,  namely  Theorem  1  and  Corollary  1. 

We  are  now  in  position  to  state  the  main  result  of  this  section.  Consider  the  chance  constrained 
problem, 

min  cTx 

s.t.  xGTe(Q)  =  {yGT|Q(H:f(y,H)0C)<e}, 
corresponding  to  a  measure  Q  G  Q.  We  will  call  x  an  (Q,  a,  r,  ^-approximate  solution  of  (37)  if 

cTx  <  inf  ctx  +  cj.  (38) 

xeAe(Q,a,r) 

Then  the  following  result  holds. 

Theorem  6  For  all  Q  G  Q  such  that  Xe(Q,a,r)  0,  SolveAmbChance  returns  an  (Q ,a,r,cv)- 
approximate  solution  with  a  probability  at  least  (1  —  MJVe“^“,1'£~0)(l  —  MS). 

The  proof  of  this  result  proceeds  along  the  lines  of  Theorem  3. 

5  Computational  Experiments 

In  this  section  we  illustrate  the  techniques  proposed  in  this  paper  by  solving  a  two-stage  net¬ 
work  design  on  the  simple  network  displayed  in  Figure  3.  The  node  s  is  a  source  node  with 
an  infinite  capacity  and  the  nodes  1,2,  and  3  are  sink  nodes  with  demands  given  by  the  vector 
d  =  (d] ,  d2,dfi)T  >  0.  For  a  given  demand  vector  d  the  network  design  problem  is  given  by 
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Figure  3:  Chance  constrained  Network  Design 

min  cru 
s.t.  Av  >  d, 
u-v  >  0, 

u,  v  >  0, 

where  the  cost  vector  c  is  assumed  to  be  strictly  positive,  —  A  denotes  the  node-arc  incidence  matrix 
of  the  network,  the  variable  u  denotes  the  capacity  on  the  arcs,  and  the  variable  v  denotes  the  flow 
on  the  network. 

Once  the  network  is  constructed,  i.e.  the  capacities  u  are  installed  on  the  arc,  we  assume  that 
it  will  be  used  over  a  reasonably  long  period  over  which  the  demand  d  can  change.  We  model 
variation  in  demand  by  assuming  that  it  is  random.  In  particular  we  assume  that  the  random 
demand  D  ~  Q  where  Q  €  Q  =  {Q  :  pp{Qo,Q)  <  /?},  Qo  =  AA(d,a2I)  is  a  multivariate  normal 
distribution  with  mean  d  and  covariance  matrix  <r2I,  and  I  denotes  the  Identity  matrix.  Note  that 
the  flow  conservation  constraints  Av  >  d  in  (39)  are  formulated  as  inequalities  instead  of  equalities 
as  is  usually  the  case  -  this  is  necessary  to  accommodate  random  demands  D. 

Although  the  capacity  cannot,  typically,  be  altered  over  the  life  of  the  network,  the  flow  v 
is  chosen  after  observing  the  realization  of  the  demand.  Thus,  the  network  design  problem  is  a 
two-stage  optimization  problem:  the  capacities  u  are  the  first  stage  variables,  and  the  flows  v  £  R" 
are  the  second  stage  variables.  The  objective  of  the  two-stage  optimization  problem  is  to  find  a 
minimal  cost  capacity  allocation  u  that  guarantees  that  at  least  (1  —  e)-fraction  of  the  random 
demand  D  can  be  feasibly  routed  in  the  resulting  network.  One  can  transform  this  network  design 
problem  into  a  chance  constrained  problem  of  the  form  (7)  by  setting 

'a!  [h' 

x  =  u,  H  =  D,  W  =  — I  ,  /(x,  H)  =  -X  ,  X  =  R” . 

1  J  [  0  _ 

In  our  computational  experiments,  we  compare  performances  of  three  different  solution  strategies: 

(a)  Deterministic  solution:  x^et  denotes  a  solution  of  the  deterministic  optimization  problem  (39) 
with  the  demand  vector  D  is  set  equal  to  its  mean  value  d. 
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(b)  Chance  constrained  solution:  xsc  is  an  output  of  Algorithm  SolveChance  . 

(c)  Ambiguous  chance  constrained  solution:  xsac  is  an  output  of  Algorithm  SolveAmbChance  . 

Instead  of  simply  verifying  the  theoretical  results  presented  in  the  previous  sections,  we  have  at¬ 
tempted  to  investigate  issues  that  we  were  not  able  to  settle  theoretically.  For  example,  we  test 
the  hypothesis  that  the  ambiguous  chance  constrained  solution  is  very  conservative  (and  protects 
against  a  set  of  measures  much  larger  than  Q)  by  using  different  test  distributions  Q. 

5.1  Algorithmic  details 

Oracles  and  Sampling  SolveChance  uses  two  oracles,  Sx  and  S-jz,  and  SolveAmbChance  ex¬ 
ploits  Sx  and  Stzp-  Since  X  =  R”  and  the  function  f(x,  h)  is  bi-afhne,  (10)  implies  that  Sx  and 
Stz  are  polynomial-time  oracles.  The  oracle  S-jzrj  requires  explicit  characterization  of  extreme  points 
of  a  polytope,  therefore  is  not  polynomial  for  some  cases.  However,  for  a  network  design  problem, 
Atamturk  and  Zhang  [2]  establish  that  S-jzf3  is  a  polynomial-time  oracle  for  some  special  networks. 

The  algorithms  SolveChance  and  SolveAmbChance  use  samples  from  AT(d,  cr2I).  Since 
SolveAmbChance  uses  more  samples  than  SolveChance,  to  avoid  generating  too  many  samples, 
we  first  run  SolveAmbChance  and  use  the  samples  generated  during  its  run  to  calculate  an  output 
of  SolveChance  . 

Stopping  criterion  The  number  of  iterations  M  in  SolveChance  and  SolveAmbChance  (see 
Figure  1  or  Figure  2)  was  chosen  large  enough  to  guarantee  convergence  in  the  underlying  Ellipsoid¬ 
like  algorithm.  In  fact,  any  stopping  rule  that  guarantees  convergence  in  the  Ellipsoid  algorithm  is 
sufficient  for  the  results  in  this  paper  to  hold. 

In  our  experiments  we  use  a  new  stopping  rule  proposed  in  [5].  Let  M  denote  the  upper  bound 
defined  in  Figure  1.  For  t  =  1  define  vt  =  1 1 d* 1 1  —  Vt  =  max{V^_i, vt},  with 

Vo  =  0,  and  pt  =  |  det(Ei)|1/n.  Then,  we  terminate  the  algorithm  at  any  iteration  t  <  M  when 

Pt  u 
r  Vt  +  uj' 

In  our  numerical  experiments  M  =  521  whereas  the  average  number  of  iterations  M  with  the  new 
stopping  rule  was  approximately  M  =  464.  This  reduction  in  the  number  of  iterations  significantly 
reduces  the  number  of  samples  generated.  In  Table  1,  the  column  labeled  Nasc(/3 )  lists  the  number 
of  samples  required  per  iteration  of  SolveAmbChance  for  a  given  (3  >  0  (Nasc( 0)  corresponds  to 
SolveChance  ),  the  columns  labeled  M  x  Nasc(/3 )  and  M  x  NaSc(P )  list  the  total  number  of  samples 
needed  with  the  old  and  the  new  stopping  rule,  respectively,  and  the  last  column  summarizes  the 
decrease  in  the  number  of  samples. 
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p 

Nasc(P) 

M  x  Nasc{(3 ) 

M  x  Nasc(f3) 

(M  -  M)  x  Nasc(P) 

0.0300 

150 

78150 

69600 

8550 

0.0200 

100 

52100 

46400 

5700 

0.0150 

86 

44806 

39904 

4902 

0.0100 

75 

39075 

34800 

4275 

0.0050 

67 

34907 

31088 

3819 

0 

60 

31260 

27840 

3420 

Table  1:  Number  of  iterations  and  samples  as  a  function  of  the  stopping  rule 


Performance  evaluation  We  assume  that  the  distribution  Q  of  the  random  demand  D  is  uncer¬ 
tain  and  is  only  known  to  belong  to  the  uncertainty  set  Q  =  {Q  :  yOp(Qo,  Q)  <  Qo  Af(d,cr2I)}. 
Since  checking  whether  a  measure  Q  €  Q  is  hard,  we  test  the  performance  of  the  solutions  using 
measures  that  are  “similar”  to  jV(d,  cr2I).  In  Section  5.2  we  discuss  performance  of  the  solutions 
when  the  test  measure  Q  is  another  Normal  distribution  with  different  parameters  and  in  Section  5.3 
we  discuss  the  results  for  Cauchy  and  Pareto  distributions. 

The  performance  of  the  solutions  was  estimated  by  out-of-sample  testing.  We  generated  50 
batches  of  1000  samples  each  (i.e.  a  total  of  50000  samples).  The  error  of  a  solution  x  with  respect 
to  a  batch  k,  k  =  1, . . . ,  50,  is  given  by 

1  1000 

errfc(x)  =  1^55  ^(1  - 

3= 1 

where  1_4  is  the  indicator  function  of  set  A  and  7£(Hj)  =  {x  |  3v  G  Rfc  s.t.  Wv  >  /(x,  Hj)}.  The 
error  of  x,  err(x),  is  defined  as  the  average  of  err^(x)  over  the  50  batches. 

Problem  parameters  The  mean  demand  vector  d  was  set  to  d  =  (3,  2,  2)T  and  standard  devi¬ 
ation  <7  was  set  to  a  =  0.5  x  10-2.  The  unit  capacity  costs  are  taken  as  cs\  =  2,  cs 2  =  0.5,  cS3  =  2, 
and  C13  =  0.1.  The  target  violation  probability  e  was  set  to  e  =  0.05.  The  other  parameters 
5  =  0.05,  r  =  10-2,  and  to  =  10~2. 

5.2  Experiments  with  the  Normal  distribution 

In  this  section  we  report  the  performances  of  the  three  solutions  when  the  test  samples  are 

drawn  from  jV(d  +  (3 ,  cr2I) ,  where  e  is  the  vector  of  all  ones  and  /3  =  0.0475,  i.e.  a  Normal 
llell 

distribution  with  a  shifted  mean.  As  indicated  in  the  previous  section,  the  training  distribu¬ 
tion  Qo  =  Af(d,u2I).  The  solutions  xsc  and  xasc  were  computed  from  one  run  of  Algorithms 
SolveChance  and  SolveAmbChance  ,  respectively.  We  will  comment  on  this  later  in  this  sec¬ 
tion. 

Table  2  displays  the  performance  of  the  three  solutions  strategies  as  a  function  of  (3 .  The 
columns  labeled  “err”  list  the  empirical  estimate  of  the  violation  probability,  the  column  labeled 
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X-asc 

X-sc 

X-det 

P 

err 

stdev 

c 

err 

stdev 

C 

err 

stdev 

C 

0.0475 

0 

0 

11.3888 

0.2131 

0.0019 

11.1810 

1.0000 

0.0000 

11.0000 

0.0450 

0 

0 

11.3790 

0.2623 

0.0014 

11.1814 

0.9999 

0.0001 

11.0000 

0.0400 

0.0000 

0.0000 

11.3277 

0.0955 

0.0014 

11.1873 

0.9997 

0.0001 

11.0000 

0.0300 

0.0000 

0.0000 

11.2912 

0.0707 

0.0014 

11.1707 

0.9983 

0.0002 

11.0000 

0.0250 

0.0000 

0.0000 

11.2798 

0.0137 

0.0006 

11.1910 

0.9954 

0.0003 

11.0000 

0.0200 

0.0001 

0.0001 

11.2509 

0.0130 

0.0005 

11.1799 

0.9894 

0.0005 

11.0000 

0.0150 

0.0001 

0.0001 

11.2358 

0.0087 

0.0003 

11.2074 

0.9775 

0.0006 

11.0000 

0.0100 

0.0002 

0.0001 

11.2162 

0.0045 

0.0003 

11.1834 

0.9577 

0.0010 

11.0000 

0.0050 

0.0005 

0.0001 

11.1922 

0.0015 

0.0002 

11.1790 

0.9233 

0.0010 

11.0000 

0.0025 

0.0003 

0.0001 

11.1916 

0.0010 

0.0001 

11.1875 

0.9033 

0.0013 

11.0000 

0.0005 

0.0005 

0.0001 

11.1759 

0.0027 

0.0002 

11.1680 

0.8782 

0.0014 

11.0000 

Table  2:  Performance  when  the  test  distribution  Q  =  A7(d  +  /$jjfp  cr2I). 

“stdev”  lists  the  standard  deviation  of  the  empirical  estimate,  and  the  column  labeled  “c”  lists 
the  cost  of  the  solution.  We  highlight  the  instances  where  empirical  estimate  “err”  violates  the 
bound  e  =  0.05  by  listing  the  value  in  bold  face.  The  constant  (3  defining  the  uncertainty  set 
Q  (see  (22))  for  the  ambiguous  chance  constraint  problem  was  set  equal  to  f3.  We  can  draw  the 
following  conclusions  from  the  results  displayed  in  Table  2. 

(i)  The  deterministic  solution  does  not  have  any  tolerance  for  variance  in  demand.  This  is 
not  surprising  since  it  completely  ignores  the  distributional  information. 

(ii)  The  chance  constrained  solution  xsc  has  more  tolerance  for  random  variations  in  demand  as 
well  as  changes  in  the  underlying  distribution  -  the  chance  constrained  solution  meets  bound 
on  the  violation  probability  e  =  0.05  for  all  values  of  (3  <  0.025.  However,  this  tolerance 
come  with  a  higher  capacity  cost.  As  f3  is  increased  beyond  the  threshold  value  of  0.025,  the 
performance  of  the  chance  constrained  solution  deteriorates  very  sharply. 

(iii)  The  ambiguous  chance  constrained  solution  never  violates  the  bound  e  =  0.05.  In  fact,  setting 
the  constant  (3  defining  Q  equal  to  (3  of  the  test  distribution  results  in  very  conservative 
solutions  -  the  violation  probability  estimates  are  considerably  smaller  than  the  allowable 
bound  of  e.  This  also  results  in  a  correspondingly  higher  capacity  cost. 

Since  setting  /3  =  (3  resulted  in  very  conservative  solutions,  we  next  investigated  whether  setting 
(3  «  (3  results  in  feasible  solutions  with  costs  comparable  to  that  of  the  chance  constrained 
solution  xsc.  Table  3  displays  the  performance  of  the  ambiguous  chance  constrained  solutions  for 
different  values  of  (3  when  (3  =  0.0475.  We  see  that  xasc(/3 )  for  all  (3  >  0.015  is  feasible  with  respect 
to  (3.  Moreover,  the  cost  of  xa<jC(0.015)  is  comparable  to  that  of  the  chance  constrained  solution 
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^asc(Z^) 

(3 

err(xosc(/3)) 

stdev(xafjC(/3)) 

c{xasc{/3)) 

0.0475 

0 

0 

11.3799 

0.0450 

0 

0 

11.3589 

0.0400 

0 

0 

11.3393 

0.0300 

0.0020 

0.0002 

11.2867 

0.0250 

0.0015 

0.0002 

11.2935 

0.0200 

0.0078 

0.0004 

11.2570 

0.0150 

0.0184 

0.0006 

11.2406 

0.0100 

0.0632 

0.0013 

11.2154 

0.0050 

0.0776 

0.0011 

11.2109 

0.0025 

0.2176 

0.0018 

11.1806 

0.0005 

0.2319 

0.0018 

11.1790 

Table  3:  Performance  when  the  test  distribution  Q  =  AA(d  +  /^]if|Ti  o'2!) ,  with  (3  =  0.0475. 


X-asci/3) 

P 

err 

stdev 

c 

0.0300 

0.0005 

0.0001 

11.3006 

0.0200 

0.0071 

0.0004 

11.2599 

0.0150 

0.0220 

0.0007 

11.2413 

0.0100 

0.0462 

0.0010 

11.2223 

0.0050 

0.1150 

0.0015 

11.1997 

X-sc 

err 

stdev 

c 

0.1853 

0.0017 

11.1851 

Table  4:  Performances  of  x.asc(/3)  and  x.ac  when  Q  =  AT(d  +  /? tj^tt ,  cr2I),  (3  =  0.0475. 

xsc  while  at  the  same  time  providing  insurance  against  all  test  distributions  with  0  <  f3  <  0.0475. 
These  results  reiterate  that  xasc(/3 )  is  very  conservative  for  a  given  level  (3;  therefore,  the  designer 
has  to  carefully  select  f3  to  protect  against  over-designing. 

The  chance  constraint  is  a  non-convex  constraint  and  the  output  of  Algorithms  SolveChance  and 
SolveAmbChance  are  both  random.  Erdogan  and  Iyengar  [12]  show  that  taking  a  convex  combi¬ 
nation  of  samples  of  a  random  solution  significantly  improves  the  performance  in  such  a  situation. 
We  test  this  result  by  studying  the  performance  of  the  solutions  xsc  and  xasc  that  are  generated  by 
taking  the  empirical  average  over  N  =  50  independent  runs.  Table  4  presents  the  results  related  to 
Xasc(/9)  and  xsc.  It  is  clear  from  the  results  in  Table  4  that  averaging  improves  the  performance. 

In  particular,  the  xasc(/3)  is  feasible  for  f3  =  0.0475  even  when  one  sets  f3  =  0.01. 
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Xasc(/^) 

P 

err 

stdev 

C 

0.0475 

0.0286 

0.0044 

11.3710 

0.0400 

0.0378 

0.0058 

11.3384 

0.0300 

0.0414 

0.0067 

11.2827 

0.0250 

0.0395 

0.0057 

11.2688 

0.0200 

0.0495 

0.0063 

11.2482 

0.0150 

0.0489 

0.0069 

11.2445 

0.0100 

0.0481 

0.0066 

11.2238 

0.0050 

0.0631 

0.0064 

11.2115 

0.0025 

0.0712 

0.0074 

11.1998 

Xsc 

err 

stdev 

c 

0.0681 

0.0081 

11.1930 

X< let 

err 

stdev 

c 

0.8722 

0.0109 

11.0000 

Table  5:  Performance  when  test  distribution  Q  is  multivariate  Cauchy  with  median  d. 

5.3  Experiments  with  other  distributions 

Table  5  displays  the  performance  of  the  three  solution  strategies  when  the  test  distribution  Q  is  a 
multivariate  Cauchy  distribution  with  median  d,  i.e.  each  D,;  is  a  Cauchy  random  variable  with 
median  d*,  i  =  1,2,3.  We  assumed  that  D,,  i  =  1,2,3  are  independent.  Once  again  we  see  that 
the  deterministic  solution  and  the  chance  constrained  solution  xsc  are  both  infeasible.  The 
ambiguous  chance  constrained  solution  is  feasible  for  all  values  of  (3  >  0.01  and  the  additional  cost 
for  the  robustness  offered  by  the  ambiguous  chance  constrained  solution  is  at  most  1.6%.  Note  that 
we  have  no  guarantee  that  the  Cauchy  distribution  belongs  to  the  uncertainty  set  Q  for  any  value 
of  fd. 

Table  6  displays  the  performances  of  the  three  solutions  when  the  test  distribution  is  Pareto 
with  variance  <r2I  and  a  new  mean  d  such  that  CDF  of  A7(d,  cr2I)  is  greater  than  the  CDF  of  the 
Pareto  for  all  d,  i.e.  each  component  of  the  chosen  Pareto  distribution  stochastically  dominates 
jV(dj,  cr2)  for  i  =  1,2,3.  The  performance  of  the  three  solutions  is  very  similar  to  the  case  when 
the  test  distribution  was  Cauchy  -  the  ambiguous  chance  constrained  solution  provides  robustness 
to  perturbation  in  the  distribution  at  a  modest  increase  in  cost. 


6  Conclusion 

In  this  paper  we  study  two-stage  convex  chance  constrained  problems.  Nemirovski  and  Shapiro  [18] 
formulated  this  class  of  problems  and  proposed  an  ellipsoid-like  iterative  solution  algorithm  for  the 
special  case  where  the  impact  function  f(x,  h)  (see  (7))  is  bi-afhne.  We  show  that  the  Algo¬ 
rithm  SolveChance  (see  Figure  1)  extends  the  results  in  [18]  to  bi-convex  f(x,  h)  in  a  fairly 
straightforward  fashion.  The  computational  complexity  of  SolveChance  as  well  as  the  quality  of 
its  output  x  depend  on  the  radius  r  of  the  largest  Euclidean  ball  that  can  be  inscribed  in  the  random 
set  Xx  (see  (11))  that  is  defined  by  the  random  set  of  linear  inequalities  generated  during  one  run 
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Xasc(/^) 

0 

err 

stdev 

C 

0.0475 

0.0013 

0.0011 

11.3842 

0.0450 

0.0016 

0.0014 

11.3626 

0.0400 

0.0036 

0.0016 

11.3297 

0.0300 

0.0080 

0.0030 

11.3004 

0.0250 

0.0082 

0.0028 

11.2753 

0.0200 

0.0108 

0.0032 

11.2443 

0.0150 

0.0241 

0.0053 

11.2402 

0.0100 

0.0255 

0.0046 

11.2230 

0.0050 

0.0513 

0.0067 

11.2221 

0.0025 

0.0513 

0.0070 

11.1911 

0.0005 

0.0666 

0.0073 

11.2259 

Xsc 

err 

stdev 

c 

0.0554 

0.0076 

11.1846 

X-det 

err 

stdev 

c 

0.9809 

0.0045 

11.0000 

Table  6:  Performance  when  Q  is  multivariate  Pareto  with  a  shifted  mean  and  variance  a 
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of  SolveChance.  Since  the  set  Xj  is  random,  selecting  r  is  difficult;  yet  SolveChance  requires  r 
as  an  input.  In  this  paper  we  provide  some  guidance  for  selecting  r.  We  show  that  the  largest  value 
of  r  is  related  to  the  degree  of  robustness  of  the  two-stage  chance  constrained  problem  -  the  more 
robust  the  problem,  the  higher  one  can  set  the  parameter  r.  This  is  reminiscent  of  results  relating 
the  condition  number  of  optimization  problems  to  their  computational  complexity  [20,  21,  13]. 

Next,  we  formulate  ambiguous  two-stage  chance  constrained  problems.  In  this  formulation,  the 
random  parameter  H  is  known  to  have  a  fixed  distribution  Q.  However,  the  decision  maker  is  only 
able  to  estimate  that  Q  belongs  to  an  uncertainty  set  Q  of  the  form  Q  =  {Q  :  pp(Q,  Qo)  <  /?},  where 
pp  denotes  the  Prohorov  metric  and  /3  is  an  exogenously  defined  constant.  We  construct  an  algo¬ 
rithm  SolveAmbChance  that  solves  the  ambiguous  two-stage  chance  constrained  problem  when 
the  impact  function  f(x,  h)  is  bi-affine  and  the  extreme  points  of  the  dual  polytope  correspond¬ 
ing  to  the  set  C  are  explicitly  known.  The  bi-affine  assumption  can  be  defended  on  the  grounds 
that  it  still  allows  one  to  model  a  wide  variety  of  applications;  however,  the  latter  assumption  is 
much  more  serious  and  cannot  be  relaxed  in  general.  We  are  currently  exploring  the  possibility 
of  replacing  the  adjustable  robust  characterization  (24)  by  a  chance  constrained  characterization. 
Let  C  =  { A  :  WrA  =  0, 1TA  =  1,  A  >  0}  and  let  P  denote  any  probability  measure  on  C.  Let 
g(x,  h,  A)  =  (ao(x)  +  Ai(x)h)7  A  +  /3||  Af  (x)A||.  Then  x  €  TZp  if,  and  only  if,  g(x,  h,  A)  <  0,  for 
all  A  G  £,  i.e.  P(A  :  g(x,  h,  A)  <  0)  =  1.  Currently,  we  are  investigating  how  one  can  relax  this 
constraint  to  P(A  :  g(x,  h,  A)  <  0)  >  1  —  e. 

In  the  context  of  ambiguous  chance  constrained  problems,  the  choice  of  the  Prohorov  metric  can 
be  justified  by  the  fact  that  it  is  the  natural  metric  for  defining  weak  convergence  of  measures.  It 
is,  however,  not  clear  how  one  should  compute  the  constant  (3.  We  propose  the  following  strategy. 
Suppose  we  assume  that  H  is  described  by  a  parametric  family  of  distributions  0  €  0. 
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Suppose  we  estimate  6  using  an  estimator  TJv(Hi, . . . ,  Hat)  and  let  Qo  =  F{Qo),  where  Oq  = 
Tjv( Hi, . . . ,  Hat).  In  the  robust  statistics  literature,  there  is  a  “breakdown  point”  e(T)  associated 
with  every  estimator  beyond  which  the  estimator  is  completely  unreliable  [14].  Heuristically,  the 
estimator  is  said  to  perform  well  for  all  measures  Q  such  that  pp(Q,Qo)  <  ^e(T).  Thus,  we  could 
set  (3  =  \e{T). 
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